Wide area networks provide an opportunity for a large number of entities to make content accessible to others. One benefit of such networks is an ever increasing and changing amount of content that is available. In order to help users find various types of content, search engines crawl wide area networks to identify new and/or changed content. This allows users to locate new content as it becomes available.
However, due to the large number of entities that provide content, quality control of new content is uneven. Some content may appear to be available, but is actually not properly formatted for viewing. Other content may be of a character that is not appropriate for viewing by all types of users. Still other problems can be related to the operation of the search engine itself. If the initial crawling and processing work of the search engine is not performed properly, the results provided to users in response to a search will be impacted.